Skip to content

Plugin DSL - #15314

Merged
oguzkocer merged 24 commits into
trunkfrom
plugin-dsl
Jun 3, 2022
Merged

oguzkocer merged 24 commits into
trunkfrom
plugin-dsl

Conversation

@oguzkocer

@oguzkocer oguzkocer commented Sep 9, 2021 •

Copy link
Copy Markdown
Contributor

This PR updates the project to use Gradle plugin DSL as part of our ongoing effort to update all our projects and also closes #15703 as outlined in this comment. This was a bit trickier than most projects, but it was not unexpected due to its size. @wzieba would you be up to reviewing this PR? You have had experience with this transition in multiple projects, so I think you're the most likely candidate to catch anything missing or wrong. As both of us experienced, there are some unexpected twists with the plugin DSL, so I am not fully confident in all the changes. (also feel free to add your thoughts in the discussion in #15703)

I tried to keep this transition as 1 to 1 as I can, but there are still a few things worth mentioning:

  • I did not port checkstyle to the plugin DSL. I tried at first, but it requires some extra research so I left it for a separate PR.
  • I had to address the issue from 🚨 Update all our dependencies to be compatible with our kotlin version #15703 where Kotlin dependencies were being resolved to 1.6.0 which happened in 320eed4 (which fixed a merge conflict issue). As far as I could tell, there were no dependencies that required Kotlin 1.6.0, but I might have missed a dynamic dependency. In any case, I think we should start forcing Gradle to resolve dependencies to the version we specify as in 9bdab52 and this was a good example why.

To test:

  • Smoke test the app

Regression Notes

  1. Potential unintended areas of impact
    Incorrect dependency version resolutions

  2. What I did to test those areas of impact (or what existing automated tests I relied on)
    N/A

  3. What automated tests I added (or what prevented me from doing so)
    N/A

PR submission checklist:

  • I have completed the Regression Notes.
  • I have considered adding accessibility improvements for my changes.
  • I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

@peril-wordpress-mobile

peril-wordpress-mobile Bot commented Sep 9, 2021 •

Copy link
Copy Markdown

You can trigger optional UI/connected tests for these changes by visiting CircleCI here.

Base automatically changed from stories-from-s3 to develop October 5, 2021 20:14
@oguzkocer
oguzkocer changed the base branch from develop to improve/build-gradle-file-format December 24, 2021 19:56
@peril-wordpress-mobile

peril-wordpress-mobile Bot commented Dec 24, 2021 •

Copy link
Copy Markdown

You can test the changes on this Pull Request by downloading the APKs:

Base automatically changed from improve/build-gradle-file-format to develop December 24, 2021 21:36
@oguzkocer
oguzkocer marked this pull request as ready for review December 24, 2021 22:03
@oguzkocer oguzkocer added this to the 19.0 milestone Dec 24, 2021
@oguzkocer
oguzkocer requested a review from wzieba December 24, 2021 22:03
Comment thread WordPress/build.gradle
testImplementation "$gradle.ext.storiesAndroidPhotoEditorPath:$storiesVersion"
implementation project(path:':libs:image-editor::ImageEditor')
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlinVersion"
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$gradle.ext.kotlinVersion"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As "dependency on the standard library (stdlib) is added automatically to each source set" we shouldn't need this or any other stdlib as direct dependency. WDYT about removing this and rest of them in other projects?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a change I've been wanting to make for a while now, but I strongly prefer making these changes in a standalone PRs especially in WPAndroid where there are quite a few modules. That's unless that change becomes a requirement for this PR.

Comment thread build.gradle Outdated
Comment on lines +57 to +60
// Don't allow Gradle to resolve the dependency to any other value
force "org.jetbrains.kotlin:kotlin-stdlib:$gradle.ext.kotlinVersion"
force "org.jetbrains.kotlin:kotlin-stdlib-common:$gradle.ext.kotlinVersion"
force "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$gradle.ext.kotlinVersion"

@wzieba wzieba Dec 27, 2021 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hichamboushaba stumbled recently on similar issue in Woo: woocommerce/woocommerce-android#5521

As he proposed similar solution, we agreed that it's not the best one as we only force version of Kotlin's stdlib, not the whole Kotlin Gradle Plugin. This means that those versions can quickly run out of sync and we believe it might cause problems - probably there's a reason why Kotlin synced those versions out of the box.

Unfortunately, we didn't find any way of forcing specific version of Kotlin Gradle Plugin and our bump is forced by other plugin (safeargs in this case).

In conclusion, I believe it's best to leave Kotlin resolution as it is and accept that external plugins can influence version of Kotlin we use. I also try always to believe that there was a reason why external plugin bumped Kotlin/Kotlin Gradle Plugin - it might not work correctly if we force standard library to lower version.

We eventually always might decide to not just bump some external plugin if we decide that Kotlin bump is dangerous. What we though gain is robust and firm build configuration, while forcing specific Kotlin stdlib version against its plugin's version seems like action against Kotlin build system design.

What are your thoughts @hichamboushaba @oguzkocer ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the ping @wzieba, as we discussed before, I agree we should keep the same version that the kotlin's plugin uses for the runtime dependencies as well.

I believe it's best to leave Kotlin resolution as it is and accept that external plugins can influence version of Kotlin we use

I'm afraid this won't be very safe, kotlin's updates need proper testing to confirm there are no hidden regressions in different Android APIs, and if we accept updates coming from external plugins, they will go unnoticed, unless there is a compile-time regression, so I prefer to hold on updating the external dependency than updating kotlin's version.

I know we didn't find a way to enforce the plugin version, but I think we can at least make the build fail when a different version than what we expect is resolved, we can probably add some constraints using a strictly version, this would make the build fail if a transitive dependency caused an update implicitly, the code would be something like this:

    constraints {
        for (module in ["stdlib", "stdlib-jdk7", "stdlib-jdk8"]) {
            implementation("org.jetbrains.kotlin:kotlin-$module") {
                version {
                    strictly gradle.ext.kotlinVersion
                }
            }
        }
    }

WDYT about this? I tested this with my navigation's update PR, and this constraint causes the build to fail, while it succeeds when I downgrade the Navigation version

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for long reply

I prefer to hold on updating the external dependency than updating kotlin's version.

Sure, my phrasing was not the best but what I meant is to accept the fact of this behaviour and not force our own standard library.

I like the way of handling you proposed. I didn't use constraints but as you say if:

we can at least make the build fail when a different version than what we expect is resolved

it would be awesome!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wzieba I understand where you're coming from, but in practice I don't think it makes sense to blindly accept a transitive dependency version. There might be a case where it's better to update our project and there might be a case where we have to force our own version. Your initial argument doesn't answer recommend a way to address the failure as far as I can tell, so I couldn't compare the trade-offs very well.

Unfortunately, we didn't find any way of forcing specific version of Kotlin Gradle Plugin and our bump is forced by other plugin (safeargs in this case).

@wzieba @hichamboushaba I think it's possible to force a specific version, let me try it out tomorrow and get back to you.

I know we didn't find a way to enforce the plugin version, but I think we can at least make the build fail when a different version than what we expect is resolved, we can probably add some constraints using a strictly version, this would make the build fail if a transitive dependency caused an update implicitly, the code would be something like this:

As far as I can tell, this is just a different syntax that'll result in the same thing because strictly will also force the same resolution. Having said that, this seems like the better syntax. If we have to force the version, and assuming the constraints get us to the solution we want, I like using this. Thank you for sharing @hichamboushaba.


@wzieba @hichamboushaba The plugin version being possibly different was a very important point, thank you for bringing it up, I unfortunately missed this idea in my initial implementation. I think it's possible to add a constraint to the plugin version resolution and once that's done for Kotlin plugin, it might be possible to avoid the extra step by removing the direct dependencies as suggested by @wzieba here. I didn't want to make that change as part of this PR as I explained in my reply, but I definitely don't mind doing it if there is an extra benefit to it.

Fwiw, I think the issue we are dealing is potentially a bug and if we can get away without using these extra constraints, we should remove it. (possibly after upgrading to Kotlin 1.6) I still think it's a good idea to have more control over how our dependencies are resolved, but I'd prefer a more general solution than just forcing one dependency to be resolved in a certain way.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, this is just a different syntax that'll result in the same thing because strictly will also force the same resolution. Having said that, this seems like the better syntax. If we have to force the version, and assuming the constraints get us to the solution we want, I like using this.

I'm not sure it's the same thing (but I may be wrong, my knowledge of gradle is very limited 🙂), the resolutionStrategy will force the dependency, so the build won't fail, even if a library is bringing a dependency with a higher version, where the constraints block will let Gradle do its work for resolving the version it needs, and it will then check it against the stated constraints, and the build will fail if they can't be respected.

I think it's possible to force a specific version, let me try it out tomorrow and get back to you.

This would be great if possible, thanks.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hichamboushaba @wzieba Just wanted to give you a heads up that it'll be a while before I can get back to this PR. I'm going to convert it back to draft and will ping you again when it's ready. Thank you for your feedback so far 🙇‍♂️

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to drop the force resolution rule since we updated Kotlin in trunk.

Comment thread settings.gradle Outdated
Comment on lines +38 to +40
if (requested.id.id == "androidx.navigation.safeargs.kotlin") {
useModule("androidx.navigation:navigation-safe-args-gradle-plugin:$gradle.ext.navComponentVersion")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't say which exactly (no mention in release notes), but after some version (e.g. 2.4.0-beta02), safeargs plugin seems to support plugin marker artifacts and there's no need to define custom resolutionStrategy

It might be valuable to add some comment on that to fix this after bumping navComponentVersion .

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 79fb6cf by updating navComponentVersion and dropping the resolution rule.

P.S: I'll open a separate PR for the navComponentVersion update before marking this PR ready for review, so it won't actually be a part of this PR,, but I wanted to reply to these comments while the changes are fresh in my mind.

Comment thread settings.gradle Outdated
Comment on lines +41 to +43
if (requested.id.id == "com.google.gms.google-services") {
useModule("com.google.gms:google-services:4.3.8")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that google-services doesn't seem to need custom resolutionStrategy - at least it works in Woo without problems. WDYT about removing this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this @wzieba! Fixed in 8329b6f.

Comment thread build.gradle
Comment on lines +4 to +5
id "androidx.navigation.safeargs.kotlin" apply false
id "com.android.library" apply false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just FYI: last week I had an attempt of fixing this by moving all Android-dependant plugins away from root project but eventually this idea failed because of how Kotlin Gradle Plugin works with Gradle API. Apparently, it seems to be not-fixable. See https://youtrack.jetbrains.com/issue/KT-31643 for details and my attempt: woocommerce/woocommerce-android#5523

Comment thread build.gradle Resolved
@oguzkocer
oguzkocer marked this pull request as draft January 7, 2022 20:09
@oguzkocer oguzkocer removed this from the 19.0 milestone Jan 8, 2022
Comment thread settings.gradle Resolved
@wpmobilebot

Copy link
Copy Markdown
Contributor

Found 1 violations:

The PR caused the following dependency changes:

 \--- project :libs:editor:WordPressEditor
-     \--- org.jetbrains.kotlin:kotlin-stdlib:1.1.4 -> 1.6.10 (*)
+     \--- org.jetbrains.kotlin:kotlin-stdlib:1.6.10 (*)

Please review and act accordingly

Without this change `:libs:editor:WordPressEditor` uses `1.6.0` version of `org.jetbrains.kotlin:kotlin-parcelize-runtime` instead of `1.6.10` which is our current `kotlinVersion`.
@oguzkocer
oguzkocer marked this pull request as ready for review June 1, 2022 17:54
@oguzkocer oguzkocer added this to the 20.1 milestone Jun 1, 2022
@oguzkocer

Copy link
Copy Markdown
Contributor Author

@ParaskP7 This PR is now ready for review. @hichamboushaba @wzieba You're also welcome to review it again if you'd like!

To summarize the recent changes:

  • I merged Update Navigation component version to 2.4.2 #16632 to trunk and then merged trunk into this branch which addressed the lint issue we had.
  • After the merge commit 4fd566e; Dependency Tree Diff CircleCI job reported that org.jetbrains.kotlin:kotlin-parcelize-runtime:1.6.0 was added to :libs:editor:WordPressEditor instead of 1.6.10 - our current kotlinVersion. This I believe is due to differences between how plugin DSL and buildscript are interpreted by Gradle. Specifically, in plugin DSL, I find that in some cases a plugin needs to be declared in both settings.gradle & root build.gradle for it to load correctly to be used by submodules. I tried this in 7d0c5b8 which fixed the issue.
  • Note that while working on 7d0c5b8, I accidentally reset my git config as I ran ./tools/dependency-tree-diff/dependency-tree-diff.sh script without cleaning up the CircleCI related commands. I ended up having to force push to fix the author of the latest commit without making any code changes. I am against force pushing after a PR is reviewed, but in this case this was the only way I know to restore the author information for the commit and since it didn't contain any code changes, I thought it to be a reasonable exception.

@ParaskP7

ParaskP7 commented Jun 2, 2022 •

Copy link
Copy Markdown
Contributor

@ParaskP7 This PR is now ready for review.

Woohoo, great, thank you so much for working on this @oguzkocer , I'll be reviewing that today! 💯

@ParaskP7 ParaskP7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👋 @oguzkocer !

I reviewed and tested this PR again, this time even more throroughtly with my general smoke testing, everything looks good and I didn't encounter any hiccups.

Needless to say, this work is VERY VALUABLE, will help us a lot going forward and THANK YOU for wrangling on it! ❤️ 🥇 💯

  1. Warning (⚠️): I didn't approve at this point just because of this single merge conflict related warning comment I raised. Other than that it is a ✅ from my side!
  2. Suggestion (💡): I would also recommend for us to trigger the UI tests before merging this to trunk, just so we reach an additional level of confidence. PS: I generally recommend doing that when we have PR such as this or with dependency update PRs, but granted, I sometimes forget to suggest that (like I forgot here) or I forget triggering that on my own PRs. 😊

PS: @hichamboushaba @wzieba do let us know if we are missing something and thank you so much for helping on this since day one! 🙇

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🚨 Update all our dependencies to be compatible with our kotlin version

5 participants